Method and apparatus for clustering news online content based on content freshness and quality of content source

ABSTRACT

Methods and apparatus are described for scoring documents in response, in part, to parameters related to the document, source, and/or cluster score. Methods and apparatus are also described for scoring a cluster in response, in part, to parameters related to documents within the cluster and/or sources corresponding to the documents within the cluster. In one embodiment, the invention may identify the source; detect a plurality of documents published by the source; analyze the plurality of documents with respect to at least one parameter; and determine a source score for the source in response, in part, to the parameter. In another embodiment, the invention may identify a topic; identify a plurality of clusters in response to the topic; analyze at least one parameter corresponding to each of the plurality of clusters; and calculate a cluster score for each of the plurality of clusters in response, in part, to the parameter.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/344,153 filed Dec. 24, 2008, which is a continuation of U.S. patentapplication Ser. No. 10/611,269 filed Jun. 30, 2003 (now U.S. Pat. No.7,568,148), which claims benefit of U.S. Provisional Patent ApplicationNo. 60/412,287 filed on Sep. 20, 2002, the disclosures of which areincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to clustering content and, moreparticularly, to clustering content based on relevance.

BACKGROUND OF THE INVENTION

There are many sources throughout the world that generate documents thatcontain content. These documents may include breaking news, humaninterest stories, sports news, scientific news, business news, and thelike.

The Internet provides users all over the world with virtually unlimitedamounts of information in the form of articles or documents. With thegrowing popularity of the Internet, sources such as newspapers andmagazines which have historically published documents on paper media arepublishing documents electronically through the Internet. There arenumerous documents made available through the Internet. Often times,there is more information on a given topic than a typical reader canprocess.

For a given topic, there are typically numerous documents written by avariety of sources. To get a well-rounded view on a given topic, usersoften find it desirable to read documents from a variety of sources. Byreading documents from different sources, the user may obtain multipleperspectives about the topic.

However, with the avalanche of documents written and available on aspecific topic, the user may be overwhelmed by the shear volume ofdocuments. Further, a variety of factors can help determine the value ofa specific document to the user. Some documents on the same topic may beduplicates, outdated, or very cursory. Without help, the user may notfind a well-balanced cross section of documents for the desired topic.

A user who is interested in documents related to a specific topictypically has a finite amount of time locate such documents. The amountof time available spent locating documents may depend on schedulingconstraints, loss of interest, and the like. Many documents on aspecific topic which may be very valuable to the user may be overlookedor lost because of the numerous documents that the user must searchthrough and the time limitations for locating these documents.

It would be useful, therefore, to have methods and apparatus forclustering content.

SUMMARY OF THE INVENTION

Methods and apparatus are described for scoring documents in response,in part, to parameters related to the document, source, and/or clusterscore. Methods and apparatus are also described for scoring a cluster inresponse, in part, to parameters related to documents within the clusterand/or sources corresponding to the documents within the cluster. In oneembodiment, the invention may identify the source; detect a plurality ofdocuments published by the source; analyze the plurality of documentswith respect to at least one parameter; and determine a source score forthe source in response, in part, to the parameter. In anotherembodiment, the invention may identify a topic; identify a plurality ofclusters in response to the topic; analyze at least one parametercorresponding to each of the plurality of clusters; and calculate acluster score for each of the plurality of clusters in response, inpart, to the parameter.

Additional aspects of the present invention are directed to computersystems and to computer-readable media having features relating to theforegoing aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate one embodiment of the inventionand, together with the description, explain one embodiment of theinvention. In the drawings,

FIG. 1 is a diagram illustrating an environment within which theinvention may be implemented.

FIG. 2 is a simplified block diagram illustrating one embodiment inwhich the invention may be implemented.

FIG. 3 is a flow diagram for ranking sources, consistent with oneembodiment of the invention.

FIG. 4 is a simplified diagram illustrating multiple categories forsources, consistent with one embodiment of the invention.

FIG. 5 is a flow diagram for defining clusters and sub-clusters,consistent with one embodiment of the invention.

FIG. 6 is a simplified diagram illustrating exemplary documents,clusters, and sub-clusters, consistent with one embodiment of theinvention.

FIG. 7A is a flow diagram for scoring clusters, consistent with oneembodiment of the invention.

FIG. 7B is a simplified block diagram illustrating one embodiment inwhich the invention may be implemented.

FIG. 8 is a flow diagram for sorting clusters, consistent with oneembodiment of the invention.

FIG. 9 is a flow diagram for ranking documents within a cluster,consistent with one embodiment of the invention.

DETAILED DESCRIPTION

The following detailed description of the invention refers to theaccompanying drawings. The detailed description does not limit theinvention. Instead, the scope of the invention is defined by theappended claims and equivalents.

The present invention includes methods and apparatus for creatingclusters. The present invention also includes methods and apparatus forranking clusters. Those skilled in the art will recognize that manyother implementations are possible, consistent with the presentinvention.

The term “document” may include any machine-readable or machine-storablework product. A document may be a file, a combination of files, one ormore files with embedded links to other files. These files may be of anytype, such as text, audio, image, video, and the like. Further, thesefiles may be formatted in a variety of configurations such as text,HTML, Adobe's portable document format (PDF), email, XML, and the like.

In the context of traditional publications, a common document is anarticle such as a news article, a human-interest article, and the like.In the context of the Internet, a common document is a Web page. Webpages often include content and may include embedded information such asmeta information, hyperlinks, and the like. Web pages also may includeembedded instructions such as Javascript. In many cases, a document hasa unique, addressable, storage location and can therefore be uniquelyidentified by this addressable location. A universal resource locator(URL) is a unique address used to access information on the Internet.

For the sake of simplicity and clarity, the term “source” refers to anentity that has published a corresponding document.

A. ENVIRONMENT AND ARCHITECTURE

FIG. 1 is a diagram illustrating an environment within which theinvention may be implemented. The environment includes a client 110, anetwork 120, and a server 130.

The client 110 may be utilized by a user that submits a query to theserver 130 and a user that retrieves information in response to thequery. In one embodiment, the information includes documents which maybe viewable by the user through the client 110. In one embodiment, theinformation is organized within clusters which are ranked, sorted, andoptimized to provide useful information to the user. The factors thatare utilized to analyze each cluster may include the recency of thedocument, the source of the document, the importance of the contentwithin the document, and the like.

In one embodiment, the client 110 may be a web browser, and the server130 includes a clustering system.

The network 120 may function as a conduit for transmissions between theclient 110 and the server 130. In one embodiment, the network 120 is theInternet. In another embodiment, the network 120 may be any type oftransmission means.

The server 130 interfaces with the client 110 through the network 120.The clustering system may be within the server 130. The clusteringsystem may include additional elements. In one embodiment, theclustering system performs a variety of functions such as analyzingclusters and documents within clusters which are explained in moredetail below and shown in reference to FIGS. 3 through 9.

FIG. 2 is a simplified diagram illustrating an exemplary architecture inwhich the present invention may be implemented. The exemplaryarchitecture includes a plurality of web browsers 202, a server device210, and a network 201. In one embodiment, the network 201 may be theInternet. The plurality of web browsers 202 are each configured toinclude a computer-readable medium 209, such as random access memory,coupled to a processor 208. Processor 208 executes program instructionsstored in the computer-readable medium 209. In another embodiment, theplurality of web browsers 202 may also include a number of additionalexternal or internal devices, such as, without limitation, a mouse, aCD-ROM, a keyboard, and a display.

Similar to the plurality of web browsers 202, the server device 210 mayinclude a processor 211 coupled to a computer-readable medium 212. Theserver device 210 may also include a number of additional external orinternal devices, such as, without limitation, a secondary storageelement, such as database 240.

The plurality of target processors 208 and the server processor 211 canbe any of a number of well known computer processors, such as processorsfrom Intel Corporation, of Santa Clara, Calif. In general, the pluralityof web browsers 202 may be any type of computing platform connected to anetwork and that interacts with application programs, such as a personalcomputer, a mobile lap top, a personal digital assistant, a “smart”cellular telephone, or a pager. The server 210, although depicted as asingle computer system, may be implemented as a network of computerprocessors.

The plurality of web browsers 202 and the server 210 may include theclustering system as embodied within the server 130 (FIG. 1). In oneembodiment, the plurality of computer-readable medium 209 and 212 maycontain, in part, portions of the clustering system. Additionally, theplurality of web browsers 202 and the server 210 are configured to sendand receive information for use with the clustering system. Similarly,the network 201 is configured to transmit information for use with theclustering system.

B. OPERATION

The flow diagrams as depicted in FIGS. 3, 5, 7A, 8, and 9 illustrate oneembodiment of the invention. In each embodiment, the flow diagramsillustrate one aspect of processing documents and/or sources ofdocuments using the clustering system.

The blocks within the flow diagram may be performed in a differentsequence without departing from the spirit of the invention. Further,blocks may be deleted, added, or combined without departing from thespirit of the invention.

A large number of documents may be electronically available for anyparticular topic through the Internet. The quality of these documentscan range from top quality journalism to unreliable reporting. Thesource of a document may predict the quality of the particular document.For example, a highly regarded source may, on average, publish higherquality documents compared to documents published by a less highlyregarded source.

The flow diagram in FIG. 3 illustrates one embodiment for rankingsources. In Block 310, a specific source is identified.

In Block 320, documents which are published by the specific source aredetected. In one embodiment, the detection of these documents publishedby a specific source may be limited to documents published within thelast month. In other embodiment, these documents may include alldocuments published by the specific source.

In Block 330, the originality of the documents are analyzed. In oneembodiment, duplicate documents which are published by the same sourceare removed. For example, duplicate documents which are published morethan once by the same source are removed so that only the originaldocument remains. Duplicate documents may be found by comparing the textof the documents published by the source. If the text of both documentsare a close match, then one of the documents may be considered aduplicate.

In another example, non-original documents which are re-published bydifferent sources are removed. For example, many news wire services suchas Associated Press and Reuters carry original documents which arere-published by other sources. These non-original documents that arere-published may be found by comparing the text of the documentre-published by one source and the text of the original documentpublished by a different source. If both documents are a close match,the original document may be determined by finding the document with theearliest publication date. For example, the first source to publishidentical documents may be considered the original author, and thecorresponding document is considered the canonical version with theremaining subsequent versions considered duplicates.

To facilitate an efficient textual analysis of comparing the documentspublished by the source with documents from other sources, the documentsfrom other sources may be limited to those which have been publishedwithin a given length of time such as within the last month.

In Block 340, the documents are analyzed for freshness. In oneembodiment, freshness may be measured by a combination of the frequencyin which the source generates new content and the speed in which adocument is published after the corresponding event has occurred. Forexample, freshness of a source can be measured by the number ofcanonical documents generated by the source over the course of X numberof days. The freshness of a source can also be measured by an averagelapse in time between an event and the publication of a documentregarding the event.

Several schemes may be utilized to identify which documents are examinedfor freshness in the Block 340. For example, the canonical documentsfrom the Block 330 are analyzed for freshness. In another example, allthe documents including both canonical and duplicates as identified inthe Block 320 are analyzed for freshness.

In Block 350, the documents are analyzed for overall quality. There aremultiple ways to analyze the document for overall quality. For example,the overall quality of the documents by the source may be judgeddirectly by humans. In another example, the overall quality of thedocuments may be indirectly assessed by utilizing circulation statisticsof the source.

In another example, the overall quality of the documents may also beassessed by utilizing the number hit or views the documents receivedwithin an arbitrary time frame. In yet another example, the overallquality of the documents may also be assessed by measuring the number oflinks pointing to the documents published by the source.

In Block 360, the source is scored based on multiple factors such as thenumber of documents published, the originality of the documents, thefreshness of the documents, and the quality of the documents. In oneembodiment, the factors such as number of documents, originality,freshness, and quality may be weighted according the desired importanceof these factors.

For example, the score of the source is stored as part of a database.The database may be located within the server 210 or accessible to theserver 210 over the network 201.

In Block 370, the source is categorized in response, at least in part,to the score from the Block 360. In one embodiment, the source may beplaced into any number categories. For example, FIG. 4 illustratescategories 410, 420, 430, and 440 for exemplary purposes. Any number ofcategories may be utilized to illustrate the different levels ofsources. The category 410 is shown as the golden source level; thecategory 440 is shown as the lowest level; and the categories 420 and430 are shown as intermediate levels. In one embodiment, the thresholdfor sources achieving the category 410 (golden source level) is targetedfor sources that carry a substantial number of canonical documents. Inone embodiment, between 5% to 10% of all sources are targeted to fallwithin the golden source level. Other targets and levels may be utilizedwithout departing form the invention.

The categorization of the source in the Block 370 may be stored as partof a database. The database may be located within the server 210 oraccessible to the server 210 over the network 201.

The flow diagram in FIG. 5 illustrates one embodiment of definingclusters, sub-clusters, and documents within the clusters andsub-clusters.

In Block 510, a plurality of sources are identified. For example, theclustering system 130 may detect the plurality of sources from one ormore categories illustrated within FIG. 4. These sources may be limitedto sources associated with specific categories such as “golden sources”.

In Block 520, documents published by the plurality of sources, asdefined in the Block 510, are identified. For illustrative purposes,documents 610, 620, 630, 640, and 650 (FIG. 6) represent exemplarydocuments published by the plurality of sources.

In Block 530, the documents identified in the Block 520 are analyzed fortheir content. For example, a topic or subject matter may beextrapolated from the analysis of the content from each of thesedocuments.

Each document may contain a document vector which describes the topic orsubject matter of the document. For example, the document vector mayalso contain a key term which characterizes the topic or subject matterof the document.

In Block 540, the documents identified in the Block 520 are grouped intoone or more clusters. In one example, the documents may be clusteredusing a text clustering technique such as hierarchical agglomerativecluster. In another example, various other clustering techniques may beutilized.

The documents may be clustered by measuring the distance between thedocuments. For example, the document vector for each document maybeutilized to measure the similarity between the respective documents. Acommon information retrieval (IR) technique such as term frequency andinverse document frequency (TFIDF) may be utilized to match documentvectors.

Various properties of the document vectors and key terms may aid inmeasuring the distance between each document. For example, the documentvectors and key terms which are identified as titles, initial sentences,and words that are “named entities” may have increased importance andmay be given increased weighting for measuring the similarities betweendocuments. Named entities typically denote a name of a person, place,event, and/or organization. Named entities may provide additional value,because they are typically mentioned in a consistent manner for aparticular event regardless of the style, opinion, and locality of thesource.

For illustrative purposes, the documents 610, 620, and 630 are showngrouped together in a cluster 660 in FIG. 6. Similarly, the documents640 and 650 are shown grouped together in a cluster 670 in FIG. 6. Inthis example, the documents 610, 620, and 630 are determined to havemore similarities than the documents 640 and 650.

In Block 550, the documents within the clusters which were formed inBlock 540 may be further refined into sub-clusters. The documents withina cluster may be further analyzed and compared such that sub-clustersoriginating from the cluster may contain documents which are even moreclosely related to each other. For example, depending on the particularcluster, each cluster may be refined into sub-clusters. Additionally,the sub-cluster may contain a sub-set of documents which are included inthe corresponding cluster, and the documents within the sub-cluster mayhave greater similarities than the documents within the correspondingcluster.

For example, the cluster 660 includes documents 610, 620, and 630 asshown in FIG. 6. The cluster 660 is further refined into sub-clusters680 and 685. The sub-cluster 680 includes the documents 610 and 620. Inthis example, the documents 610 and 620 within the sub-cluster 680 aremore closely related to each other than the document 630 which isisolated in a different sub-cluster 685.

The comparison of document vectors and key terms and other techniques asdiscussed in association with the Block 540 may be utilized to determinesub-clusters.

In Block 560, the documents within a sub-cluster are checked for theirlevel of similarity. If there are multiple documents within thesub-cluster, and the documents are not identical enough, then thedocuments within the sub-cluster may be further refined and may beformed into lower level sub-clusters in Block 550. When sub-clusters areformed into lower level sub-clusters, a stricter threshold is utilizedto group sets of identical or near identical documents into these lowerlevel sub-clusters.

For the sake of clarity, lower level sub-clusters are not shown in FIG.6. However, forming lower level sub-clusters from a sub-cluster isanalogous to forming the sub-clusters 680 and 685 from the cluster 660.

If there are multiple documents within the sub-cluster, and thedocuments are identical enough, then the canonical document isidentified by the earliest publication time and the remaining documentsmay be considered duplicates in Block 570.

The flow diagram in FIG. 7A illustrates one embodiment of scoringclusters. In Block 705, a topic is identified. In one embodiment, thetopic is customized to a user. For example, the topic may be in the formof a query of key word(s) initiated by the user. The topic may also beidentified from a personalized page belonging to the user. The topic mayalso be selected from a generic web page which allows users to selectdifferent topic categories of interest.

In Block 710, clusters that are matched to the topic are identified. Forexample, clusters which are similar to the topic may be identified.

The identified clusters may be scored on various factors within theBlocks 715, 720, 725, 730, and 735. In Block 715, these clusters arescored based on the recency of canonical documents within each cluster.For example, a cluster with the most recent canonical documents may bescored higher than other clusters.

The recency of canonical documents may be computed for the cluster byusing a weighted sum over the original documents within the cluster. Inone example, a weighting scheme is utilized where a higher weight isgiven for fresher and more recent document.

In one embodiment, each document within a cluster may be sorted andassigned a bin that corresponds with the age of the document. Each binis configured to accept documents within a time range and correspondswith a specific weighting factor. The specific weighting factorcorresponds with the time range of the documents within the bin. In oneembodiment, the weighting factor increases as the time range correspondswith more recent documents.

FIG. 7B illustrates the use of bins relative to the weighted sum incomputing the recency of coverage. Bins 760, 765, 770, 775, and 780 areshown for exemplary purposes. Additional or fewer bins may be utilizedwithout departing from the scope of the invention.

For example, the bin 760 may have a time range which includes documentsthat have aged less than 60 minutes. In this example, the documentswithin the bin 760 are assigned a weighting factor of 24. The bin 765may have a time range which includes documents that have aged more than60 minutes and less than 2 hours. In this example, the documents withinthe bin 765 are assigned a weighting factor of 20. The bin 770 may havea time range which includes documents that have aged more than 2 hoursand less than 4 hours. In this example, the documents within the bin 770are assigned a weighting factor of 15. The bin 775 may have a time rangewhich includes documents that have aged more than 4 hours and less than24 hours. In this example, the documents within the bin 775 are assigneda weighting factor of 3. The bin 780 may have a time range whichincludes documents that have aged more than 24 hours. In this example,the documents within the bin 780 are assigned a weighting factor of −1.

Different weighting factors and time ranges may be utilized withoutdeparting from the scope of the invention.

In use, the cluster score may be calculated, in part, by multiplying thenumber of documents within each bin with the corresponding weightingfactor and summing the results from each of these multiplications. Forexample, a cluster contains documents with the following distribution ofdocuments: bin 760 contains 2 documents, bin 765 contains 0 documents,bin 770 contains 1 document, bin 775 contains 5 documents, and bin 780contains 10 documents. A sample calculation for a cluster score, basedon the sample weighting factors and document distribution as describedabove, is shown in Equation 1.sample cluster score=3×24+0*20+1*15+5*3+10*−1   Equation 1

In Block 720, the clusters may be scored based on the quality of thesources that contribute documents within each cluster. For example, thesources may be ranked on an absolute grade. According to this example, awell-known source such as the Wall Street Journal may be ranked higherthan a local source such as the San Jose Mercury News regardless of thetopic.

The importance of a source may also be computed based on the notorietyof the source. For example, the source may be computed based on thenumber of views or hits received by the source. In another example, theimportance of a source may be computed based on the circulationstatistics of the source. In yet another embodiment, the importance ofthe source may be based on the number of links for each of the documentspublished by the source and within the cluster.

In addition, the quality of the source may be based on the importance ofthe source relative to the particular topic. For example, with andocument relating to a news story, sources such as CNN, New York Times,Los Angeles Times, and Reuters may be included with a top tier sourcecategory; sources such as XYZ News and Any City Times may be included ina second tier source category; and sources such as local newsorganizations may be included in a third tier source category.

The importance of the source may depend, at least in part, on thesubject matter of the particular document and may change with eachunique topic. For example, with a query and associated documentsrelating to a local news story, the local news organizations where theevent which is identified in the query is located may be included withinthe top tier source category. These local news organizations maytypically be included within the third tier source category for anational event/story, however due to the local nature of the eventrelated to the query, these local news organizations may be elevated tothe top tier source category for this particular query based ongeography.

The importance of the source may be proportional to the percentage ofdocuments from the source which match the subject matter of the topic.For example, if the topic relates to the subject of “music”, then asource which may be considered important is MTV, because the majority ofMTV's prior documents are related to music. The importance of thesources may also change over a period of time for the same topicdepending on the subject matter of the documents.

In Block 725, the clusters are scored based on the number of canonicaldocuments contained within each cluster. For example, the cluster may bescored higher when there are more canonical documents within thecluster.

In Block 730, the clusters may be scored based on the number ofsub-clusters within each cluster. For example, the number ofsub-clusters within a cluster may be utilized to measure the amount ofdiversity of documents within the cluster.

In Block 735, the clusters may be scored based on a match between thesubject of the cluster and a set of predetermined topics. For example,the set of predetermined topics are deemed as important topics such asworld news, national news, and the like. A cluster may be scored higherwhen there is a match between the subject of the cluster and one of theset of predetermined topics.

In Block 740, each of the clusters may be scored according to parameterssuch as recency of documents, quality of sources, number of documents,number of sub-clusters, and important topics. In one embodiment, thescore for each clusters may be calculated by adding up the scoresattributed by the parameter analyzed in the Blocks 715, 720, 725, 730,and 735. In one embodiment, these scores may be stored within thedatabase 140. In addition, the clusters are sorted by score.

The flow diagram in FIG. 8 illustrates one embodiment of sortingclusters. In Block 810, a topic is identified. In one embodiment, thetopic is customized to a user. For example, the topic may be in the formof a query of key word(s) initiated by the user. The topic may also beidentified from a personalized page belonging to the user. The topic mayalso be selected from a generic web page which allows users to selectdifferent topic categories of interest.

In Block 820, clusters that are already sorted by the Block 780 areidentified.

In Block 830, centroids may be computed for each cluster. Typically, thecentroid uniquely describes the topic or subject matter of the cluster.In one example, the centroid is computed by averaging individual termvectors from the documents contained within the cluster. The termvectors may include a weighted set of terms.

In Block 840, the centroids of the sorted clusters may be compared withthe centroids of previously viewed clusters. If the centroid of aparticular cluster is similar to the centroids of the previously viewedclusters, this particular cluster may be rated lower than other sortedclusters which are not similar to the centroids of previously viewedclusters. The similar particular cluster may be rated lower thandissimilar sorted clusters, because the user has already vieweddocuments which are related to the similar particular cluster.

In Block 850, the clusters identified in the Block 820 may be sortedbased on the comparison between the centroids of these clusters and thecentroids of previously viewed clusters.

The flow diagram in FIG. 9 illustrates one embodiment of ranking anddisplaying documents within a cluster. In Block 910, a cluster isidentified. In Block 920, duplicate documents within the cluster may beremoved. For example, canonical documents remain within the clusters.

In Block 930, the documents within the cluster may be rated for recencyand/or content. With regards to recency for example, if a document isten (10) hours old, the document is assigned a recency score of ten (10)hours. In assigning a recency score, the measurement of recency may bedefined as the difference in time between the present time and the timeof publication. In another embodiment, the measurement of recency may bedefined as the difference in time between publication of the documentand the time of a corresponding event. In yet another embodiment, themeasurement of recency may be defined as the difference in time betweenthe present time and the time of an event corresponding to the document.

In addition to recency, the documents may also be scored according tothe length of a document, the title of a document, and genre of adocument. For example, in one embodiment, the longer the document is,the higher the particular document may be scored.

The title of the document may be analyzed in a variety of ways to aidein scoring the document. For example, the length of the title may beutilized to score the document with a longer title scoring a higherscore.

In another example, the title may be searched for generic terms such as“News Highlights”, “News Summary”, and the like. In one embodiment, thehigher percentage of generic terms used with the title, the lower thedocument may be scored. By the same token, use of proper nouns in thetitle may increase the score of the document. In yet another example,the words within the title may be compared with the centroid of thecluster which contains the document. In one embodiment, the score of thedocument is higher if the title contains words that overlap or match thecentroid of the cluster.

Based on the content of the document, the document may belong to aspecific genre such as “op/ed”, “investigative report”, “letter to theeditor”, “news brief”, “breaking news”, “audio news”, “video news”, andthe like. The score of the document may increase if the genre of thedocument matches the genre of a query. In one embodiment, the queryspecifies a particular genre. In another embodiment, the query includesthe genre which passively specified by the user as a preference.

In Block 940, the documents within the cluster may be rated for qualityof the corresponding source. Rating the quality of the correspondingsource is demonstrated in the Block 350 (FIG. 3).

The measurement of recency for a document (as described in the Block930) may be taken into account with the quality of the correspondingsource.

For example, if a document is assigned a recency score of ten (10) hoursand the corresponding source is considered a golden source, then amodified recency score is ten subtracted by X (10−X) hours where X is aselected value based on the quality of the source. In this example, themodified recency score is less than the original recency score, becausethe corresponding source is considered a golden source.

In another example, if a document is assigned a recency score of ten(10) hours and the corresponding source is considered a lowest category,then a modified recency score is ten added by Y (10+Y) hours where Y isa selected value based on the quality of the source. In this example,the modified recency score is greater than the original recency score,because the corresponding source is considered a lowest category.

In Block 950, the documents may be sorted by recency of the document andquality of the source. The documents may be sorted by the modifiedrecency score as shown in the Block 940. In one embodiment, thedocuments are sorted with the most recent documents listed first.

In Block 960, the most recent document within each sub-cluster of thecluster is identified and included as part of a displayed list ofdocuments. The displayed list of documents utilizes an order with themost recent document listed first according to the modified recencyscore.

In Block 970, the documents within the displayed list of documents asdescribed in the Block 960 are weighted by the number of documentswithin the corresponding sub-cluster. In one embodiment, the modifiedrecency score is further modified. For example, the more documentswithin a particular sub-cluster may increase the importance of thedocuments within the sub-cluster. In other words, the individualdocuments within the displayed list of documents are weighted based onthe number of documents within the sub-cluster.

In one example, a first and second document may each have a modifiedrecency score of ten (10) hours. However, the first document is within asub-cluster which includes twenty (20) documents, and the seconddocument is within a sub-cluster which includes ten (10) documents.According to one embodiment, the first document has a new modifiedrecency score of eight (8) hours based on the weighting of the number ofdocuments within the sub-cluster. The second document has a new modifiedrecency score of twelve (12) hours based on the weighting of the numberor documents within the sub-cluster. Accordingly, the first document hasa lower modified recency score than the second document and may be in ahigher priority position to be viewed or displayed.

In Block 980, the displayed list of documents is shown to the user. Thedocuments within the displayed list of documents may be shown to theuser based on the modified recency score as formed in the Block 960. Thedocuments within the displayed list of documents may be shown to theuser based on the modified recency score as modified in the Block 970.

C. CONCLUSION

The foregoing descriptions of specific embodiments of the invention havebeen presented for purposes of illustration and description. Forexample, the invention is described within the context of documents asmerely one embodiment of the invention. The invention may be applied toa variety of other electronic data such as pictures, audiorepresentations, graphic images, and the like.

For the sake of clarity, the foregoing references to “browser” are afigurative aid to illustrate a particular device which is utilized by aspecific user.

They are not intended to be exhaustive or to limit the invention to theprecise embodiments disclosed, and naturally many modifications andvariations are possible in light of the above teaching. The embodimentswere chosen and described in order to explain the principles of theinvention and its practical application, to thereby enable othersskilled in the art to best utilize the invention and various embodimentswith various modifications as are suited to the particular usecontemplated. It is intended that the scope of the invention be definedby the Claims appended hereto and their equivalents.

What is claimed:
 1. A method comprising: identifying, by one or moreprocessors, online content, the online content being associated with adocument; calculating, by one or more processors, a first score for theonline content, based on a measure of freshness of the online content,the online content including content relating to an event, and themeasure of freshness being based on a first time when the event occurredand a second time when the online content was published; determining, bythe one or more processors, a second score for the online content basedon a quality of a source that published the online content, the sourcebeing an entity that publishes documents; ranking, by the one or moreprocessors, the online content based on the first score and the secondscore; and providing, by the one or more processors, the ranked onlinecontent for display.
 2. The method of claim 1, where the measure offreshness is based on an amount of time between the first time and thesecond time.
 3. The method of claim 1, where the measure of freshness isbased on a frequency in which the source of the online content publishesnew online content in a particular time period.
 4. The method of claim1, where the second score is based on circulation statistics of thesource.
 5. The method of claim 1, where the second score is based on anumber of views of the online content that occur within a particulartime frame.
 6. The method of claim 1, further comprising: determiningwhether duplicates of the online content exist that were publishedbefore the online content was published by the source; and calculatingthe second score after determining whether the duplicates of the onlinecontent exist.
 7. The method of claim 1, further comprising: ranking thesource, among other sources, based on the second score.
 8. A systemcomprising: one or more processors to: identify online content, theonline content being associated with a document; calculate a first scorefor the online content based on a measure of freshness of the onlinecontent, the online content including content relating to an event, andthe measure of freshness being based on a first time when the eventoccurred and a second time when the online content was published;determine a second score for the online content based on a quality of asource that published the online content, the source including an entitythat publishes documents; rank the online content based on the firstscore and the second score; and provide the ranked online content fordisplay.
 9. The system of claim 8, where the measure of freshness isbased on an amount of time between the first time and the second time.10. The system of claim 8, where the measure of freshness is based on afrequency in which the source of the online content publishes new onlinecontent in a particular time period.
 11. The system of claim 8, wherethe second score is based on circulation statistics of the source. 12.The system of claim 8, where the second score is based on a number ofviews, of the online content that occur within a particular time frame.13. The system of claim 8, where the one or more processors are furtherto: determine whether duplicates of the online content exist that werepublished before the online content was published by the source; andcalculate the second score after determining whether the duplicates ofthe online content exist.
 14. The system of claim 8, where the one ormore processors are further to: rank the source, among other sources,based on the second score.
 15. A non-transitory computer-readable mediumstoring instructions, the instructions comprising: one or moreinstructions that, when executed by one or more processors, cause theone or more processors to: identify online content, the online contentbeing associated with a document; calculate a first score for the onlinecontent based on a measure of freshness of the online content, theonline content including content relating to an event, and the measureof freshness being based on a first time when the event occurred and asecond time when the online content was published; determine a secondscore for the online content based on a quality of a source thatpublished the online content, the source including an entity thatpublishes documents; rank the online content based on the first scoreand the second score; and provide the ranked online content for display.16. The non-transitory computer-readable medium of claim 15, where themeasure of freshness is based on an amount of time between the firsttime and the second time.
 17. The non-transitory computer-readablemedium of claim 15, where the measure of freshness is based on afrequency in which the source of the online content publishes new onlinecontent in a particular time period.
 18. The non-transitorycomputer-readable medium of claim 15, where the second score is based oncirculation statistics of the source.
 19. The non-transitorycomputer-readable medium of claim 15, where the second score is based ona number of views, of the online content that occur within a particulartime frame.
 20. The non-transitory computer-readable medium of claim 15,where the instructions further comprise: one or more instructions todetermine whether duplicates of the online content exist that werepublished before the online content was published by the source; and oneor more instructions to calculate the second score after determiningwhether the duplicates of the online content exist.